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Abstract. While there have been a number of studies about the efficacy 
of methods to find exact Nash equilibria in bimatrix games, there has 
been little empirical work on finding approximate Nash equilibria. Here 
we provide such a study that compares a number of approximation meth¬ 
ods and exact methods. In particular, we explore the trade-off between 
the quality of approximate equilibrium and the required running time to 
find one. We found that the existing library GAMUT, which has been 
the de facto standard that has been used to test exact methods, is insuf¬ 
ficient as a test bed for approximation methods since many of its games 
have pure equilibria or other easy-to-find good approximate equilibria. 
We extend the breadth and depth of our study by including new inter¬ 
esting families of bimatrix games, and studying bimatrix games upto size 
2000 x 2000. Finally, we provide new close-to-worst-case examples for the 
best-performing algorithms for finding approximate Nash equilibria. 


1 Introduction 

The computation of Nash equilibria in bimatrix games is one of the central topics 
in game theory, which has received much attention from a theoretical point of 
view. It has been shown that the problem of finding a Nash equilibrium is PPAD- 
complete [5,3], which implies that we are unlikely to find a polynomial-time 
algorithm for this problem. Naturally, this has led to a line of work studying the 
complexity of finding approximate Nash equilibria [6, 2,19,11, 7]. 

Most of the work on approximation algorithms has focussed on additive ap¬ 
proximations, where two different notions are used. An e-Nash equilibrium re¬ 
quires that both players achieve an expected payoff that is within e of a best re¬ 
sponse, while the stronger notion of e-uiell supported Nash equilibrium (e-WSNE) 
requires that both players only place probability on strategies that are within 
e of a best response. The current state of the art for e-Nash equilibria is the 
algorithm of Tsaknakis and Spirakis [19], which finds a 0.3393-Nash equilibrium 
in polynomial time, and the current state of the art of e-WSNE is the algorithm 
of Fearnley et al. [7], which finds a (| — 0.00591)-WSNE in polynomial time. 

So far, most of the work on approximate equilibria has been theoretical in 
nature. The goal of this paper is to answer the following question: Are approx¬ 
imate equilibria relevant to the problem of solving bimatrix games in practice? 
To answer this, we must study several related questions. 

— Firstly, how good are the algorithms for finding exact Nash equilibria in practice? 
If they are good enough, then there is no need for approximation. Otherwise, 
how much faster are the approximation algorithms? 



Secondly, what quality of approximation do the approximation algorithms pro¬ 
vide in practice? If the best theoretical guarantee of a 0.3393-Nash equilibrium 
is not beaten in practice, it is unlikely to be useful. 

Finally, is there a trade off between running time and approximation? We have 
a wide variety of approximation algorithms, from those that solve a single linear 
program, to those that perform complicated gradient descent procedures. Do fast 
algorithms generally produce worse approximate equilibria? Should our desired 
quality of equilibrium impact our choice of algorithm? 

Our contribution. While there have been several empirical studies on computing 
exact equilibria [4,8,15,16], the empirical work for approximate equilibria has 
so far been limited to a paper [20] that evaluates the algorithm of Tsaknakis and 
Spirakis (TS) [19], and one that looks exclusively at symmetric games [12]. We 
address this by performing a comprehensive study of approximation algorithms. 
We compare the performance of five different algorithms for finding approximate 
equilibria on 15 different types of game. Moreover, we include two algorithms for 
finding exact equilibria: the Lemke-Howson algorithm and support enumeration. 

This allows us to answer the questions posed earlier. Firstly, we find that ap¬ 
proximation algorithms can tackle instances that exact algorithms cannot. With 
a timeout of 15 minutes, we found that exact algorithms were mostly unable 
to solve instances of size 1000 x 1000, whereas approximation algorithms could 
easily tackle instances of size 2000 x 2000. Secondly, we find that approximation 
algorithms often perform much better than their theoretical worst case guaran¬ 
tees: in agreement with the experimental study of Tsaknakis et al. [20], we find 
that the TS algorithm often finds 0.01-Nash equilibria or better. In answer to 
the third question: while our data shows that the TS algorithm clearly wins in 
terms of quality of approximation, it is usually the slowest, and if we only require 
weaker approximate equilibria, then other algorithms can find one faster. 

To obtain our results, we tested the algorithms on a wide variety of games. 
Previous work on exact equilibria has typically used the GAMUT library [14]. 
However, almost all of the games provided by GAMUT have exact pure Nash 
equilibria, so using only these games could skew our results. For example, the 
work of Porter et al. [15] concluded that support enumeration typically outper¬ 
forms the Lemke-Howson algorithm, based on the fact that support enumeration 
can quickly find the pure equilibria in the games provided by GAMUT. 

There are many practical applications of game theory where all equilibria 
use mixed strategy profiles. Our second contribution is to define several natural 
classes of games that do not have pure Nash equilibria. All of our game generators 
and algorithm implementations are open source and freely available 1 . Our results 
show that algorithms perform very differently on these games, so they should be 
included in any future study of the practical aspects of computing equilibria. 

The excellent performance of the TS algorithm raises the question of whether 
the upper bound of 0.3393 on the quality of approximation of this algorithm is 
tight. We used a genetic algorithm to search for worst-case examples, and we 
found a 5 x 5 bimatrix game in which the TS algorithm gives a 0.3385-Nash 
equilibrium, which shows that the performance guarantee is essentially tight. 
We applied this technique to the algorithm of Fearnley et al. [7] for finding e- 

1 http://bimatrix-games.github.io/ 



WSNE, which likewise had no good theoretical lower bound. However, we were 
only able to find an example for which the algorithm gives a 0.4799-WSNE, 
which is far from the upper bound of (| —0.00591375). Finally, to test the limits 
of approximation techniques, we ran the same procedure against the combination 
of our three best approximation algorithms (the TS algorithm, the algorithm of 
Bosse et al. [2], and the best pure strategy pair). Here the fitness function was the 
minimum of the approximations provided by the algorithms. We found a game 
for which all of the algorithms gave no better than a 0.3189-Naslr equilibrium, 
which is relatively close to the theoretical upper bound of 0.3393, and indicates 
that new techniques will be needed to advance the theoretical state of the art. 

Related work. For exact equilibria, Porter et al. [15] use GAMUT to com¬ 
pare support enumeration (SE) with the Lemke-Howson (LH) algorithm. They 
showed that SE performs well when compared to LH, because many of the games 
in the library have small support equilibria. They also highlight random games 
and covariant games as the most challenging GAMUT games for SE and LH, 
which our results also confirm. Most of their experiments considered games of size 
600 x 600, but they did consider random games at game sizes up to 1000 x 1000. 

Sandholm at al. [16] describe four ways of solving games via mixed integer 
programming (MIP). Experiments were carried out games of size 150 x 150 
provided by GAMUT. It was found that MIP performed better than LH but 
was outclassed by SE. 

Codenotti et al. [4] studied of LH algorithm, on random and covariant games, 
where it was found that LH has a running time of 0(n") for n x n covariant 
games. They presented a heuristic which involves running different LH paths 
in parallel, which showed an improvement over LH for random games, but not 
covariant games. They looked at games of size up to 1000 x 1000. 

Tsaknakis et al. [20] performed an experimental analysis of their algorithm 
(TS) for finding a 0.3393-Nash equilibrium in polynomial time. They studied 
games of size 100 x 100, and they also constructed games with no small support 
equilibria, to prevent easy solutions. It was found that TS always finds a 0.015- 
Nash equilibrium or better. We confirm the result that in general TS performs 
well, however among some of our games TS only finds a 0.14-Nash equilibrium. 

Gatti et al. [8] evaluated the performance of LH, MIP and SE. They intro¬ 
duced a number of heuristics, and compared their performance on the games 
from GAMUT. They found that none of the methods was superior for all games. 
They did look at the quality of approximation achieved by their heuristics and 
algorithms, but the largest games they looked at were of size 150 x 150. 

2 Experimental Setup 

Algorithms. Now, we describe the algorithms that are studied in this paper. For 
the computation of exact equilibria, we study the following two algorithms. 

— LH. The Lemke-Howson algorithm is a widely-used pivoting algorithm [13]. It 
has exponential worst case behaviour [17], and it is PSPACE-lrard to compute the 
equilibrium that it finds [10]. We use the implementation from [4], modified so 
that degeneracy is resolved by the lexicographic minimum ratio test. 



SE. Support enumeration is brute force algorithm that, for every possible pair 
of supports, solves a system of linear equations to check for an equilibrium. Our 
implementation goes through supports from small to large cardinality. 

There is a wide variety of algorithms for finding e-Nash equilibria, and we 
have implemented the following (polynomial-time) algorithms: 

Pure. This algorithm checks all pure strategy profiles and returns one that gives 
the best e-Nasli equilibrium. 

DMP. This algorithm was given by Daskalakis et al. [6]. It finds a 0.5-Nash 
equilibrium using a very simple approach that starts with an arbitrary pure 
strategy, and then makes two best response queries. 

BBM1. This is the first of two algorithms given by Bosse et al. [2]. It finds a 
0.3819-Nasli equilibrium. Given a bimatrix game ( R,C ), BBM1 solves the zero- 
sum game (R—C,C—R) using linear programming. Then it proceeds in a similar 
manner to DMP, but uses the LP solution as the initial strategy. 

BBM2. This is the second of the two algorithms given by Bosse et al. [2], It 
finds a 0.3639-Nash equilibrium. It is an adaptation of BBM1 that contains some 
extra steps to deal with cases where the first algorithm performs poorly. 

TS. This algorithm was given by Tsaknakis and Spirakis [19]. It finds a (0.3393+ 
(5)-Nasli equilibrium, where d is an arbitrary positive constant. The algorithm 
uses gradient descent over the space of mixed strategy profiles. The objective 
function is the quality of approximate Nash equilibrium. The algorithm finds a 
stationary point. If the stationary point is not a (0.3393 + <5)-Nash equilibrium, 
then it can be used to find a second point that is a (0.3393 + <5)-Nash equilibrium. 
To investigate the dependence of the algorithm on 5, we use two versions with 
S = 0.2 and <5 = 0.001. We refer to these as TS2 and TS001, respectively. At 
the end of Section 3, we analyze the effect of the choice of 5. 

There has been comparatively less study of algorithms to find approximate 
well-supported Nash equilibria. We implemented the following two algorithms: 

KS. This algorithm was given by Kontogiannis and Spirakis, and it finds a 
|-WSNE [11], The algorithm first checks all pure strategy profiles in order to 
determine if there is a pure |-WSNE. If not, then the algorithm solves the same 
zero-sum game as BBM1/BBM2, and the equilibrium of this game is a |-WSNE. 
KS+. This algorithm gives an improved approximation guarantee compared to 
KS [7] of (| —0.00591). It combines the KS algorithm with two extra procedures: 
one that finds the best WSNE with 2x2 support, and one that finds the best 
WSNE on the supports from an equilibrium to the KS zero-sum game. 

Game classes. We now describe the classes of games used in our study. In order 
to have a consistent meaning of approximation guarantees, all games are scaled 
to have payoffs in [0,1]. Firstly, we used games provided by the GAMUT library. 
We used every class of games in GAMUT that could be scaled indefinitely. We 
eliminated the classes of games that have fixed size, and we were also forced 
to eliminate some classes of games because their generators either crashed or 
produced invalid games when asked to produce games with more than 1000 
strategies per player, which included BidirectionalLEG and RandomLEG. We 
were left with the games shown below. 



BertrandOligopoly CournotDuopoly CovariantGame 

GrabTheDollar GuessTwoThirdAve MinimumEfFortGame 

TravelersDilemma RandomGame WarOfAttrition 

In covariant games each pure strategy profile is drawn from a multivariate 
normal distribution with covariance p. When p = 1 we have a coordination game 
and when p = — 1 we have a zero-sum game. Previous work [15,4] indicates that 
these games are easy to solve when p > 0, with the hardest games in the range 
[—0.9, —0.5]. We study 5 classes of games CovariantGame-p for p = 1,3, 5, 7,9 
where p = —0.1, —0.3, —0.5, —0.7, —0.9 respectively. 

With the exception of covariant and random games, the other bimatrix game 
classes provided by GAMUT have pure equilibria, and are therefore easily solved 
by support enumeration. In order to broaden our study, we chose to implement 
generators for the following games, which generally do not have pure equilibria. 

Non-zero sum colonel Blotto Games. The players have an equal number 
of soldiers T that must be assigned simultaneously to n hills. Each player has a 
value for each hill that he receives if he assigns strictly more soldiers to the hill 
than his opponent (ties are broken uniformly at random.) Each player’s payoff is 
the sum of the value of the hills won by that player. To avoid pure equilibria, the 
hill values are drawn from a multivariate normal distribution with covariance 
of p > 0. In our experiments, we study families of games with n = 3,4 and 
p = 0.5,0.7, 0.9, which we denote by Blotto-n-p for p = 5, 7,9, respectively. The 
number of soldiers T was varied in order to generate a scalable family of games. 
Ranking Games [9]. Each player chooses an effort level with an associated cost 
and score. A prize is given to the player with the higher score, or is split in the 
case of a tie. The payoff of a player is the value of the prize minus the cost of 
the effort. We generated scores and costs as increasing step functions of effort 
with random step sizes. We denote these games by Ranking. 

SGC games. Sandholm et al. [16] also noted that most GAMUT games have 
small support equilibria. They introduced a family of games where, in all equi¬ 
libria, both players use half of their actions. We denote their games as SGC. In 
these game the only equilibrium in a (2k — 1) x (k — 1) game has support sizes k 
for both players, which makes these games hard for support enumeration. 
Tournament Games [1], Starting with a random tournament, an asymmetric 
bipartite graph is constructed where one side corresponds to the nodes of the 
tournament, and the other corresponds to subsets of nodes. The bipartite graph 
is transformed into a win-lose game where the actions of each player are the 
nodes on their side of the graph. We denote these games as Tournament. 

Unit Vector Games (UVG) [18]. The payoffs for the column player are chosen 
randomly from the range [0,1], but for the row player each column j contains 
exactly one 1 payoff with the rest being 0. In order to avoid pure equilibria, we 
generated these games by placing the Is uniformly at random in the rows that 
do not generate a pure equilibrium. We denote these games as Unit. 

Some of these games are not square. So, in our results we use instances that 
have roughly the same name number of payoffs as the corresponding square 
games, e.g., we compare 100 x 100 games to non-square games with roughly 
10000 payoff entries. 



Unlike other studies [8,16], we do not include the exponential-time examples 
for LH devised by Savani and von Stengel [17]. They are not suitable for an 
experimental study on approximate equilibria because the games have a number 
of large payoffs, so when they are normalised to the range [0,1], almost all payoffs 
in the game are close to 0. Hence these games are very easy to approximate. For 
example, the 30 x 30 instance has a pure 1.63 x 10 _lo -WSNE. Furthermore, for 
instances larger than 16 x 16, these games exhaust the precision of floating point. 

Implementation details. All implementations are written in C. We used CPLEX 
to solve the linear programs used in some of the algorithms. For our runtime 
results, we only measured the amount of time spent by the solver, and discarded 
the time taken to read the game from its input file. Our experiments were carried 
out on a cluster of 8 identical machines running Scientific Linux 6.6, which each 
have an Intel Core i5-2500k processor clocked at 3.30GHz with 16GB of RAM. 

To verify our results, we implemented three programs that compute the qual¬ 
ity of exact, approximate, and well-supported Nash equilibria, respectively. All 
of these programs carry out their calculations in exact arithmetic. Our exact 
equilibrium checker takes a pair of supports, and checks whether there exists a 
Nash equilibrium on these supports. Both of the approximate checkers take a 
mixed strategy profile, and output the value of e that this profile achieves. 

3 Experimental Results 

Exact Algorithms. We tested LH and SE against our library of games with a 
timeout of 15 minutes. Table 1 part A shows the percentage of games that were 
solved by these two algorithms for various game sizes. We have divided the 
games into three classes. Firstly we have GAMUT games that always have pure 
equilibria. As expected, SE performs well on these games, while LH is also able 
to tackle the majority of instances. Secondly, we have the GAMUT games that 
do not always have pure equilibria. Both algorithms performed very poorly for 
covariant games, which is in agreement with previous studies. Finally, we have 
the games that we proposed. Both algorithms struggle with the games in this 
class, which supports the idea that GAMUT’S existing library does not give a 
comprehensive picture of possible games. Ranking games provide an interesting 
case that differentiates between LH and SE: these games only had equilibria with 
medium sized support so SE was hopeless, however LH was able to solve these 
games using a linear number of pivots. In conclusion, our results show that exact 
methods are inadequate for the 2nd and 3rd classes of games, so it is for these 
games that we are interested in the performance of approximation methods. 

Approximation algorithms. Table 1 part B shows the running time and quality 
of approximation, respectively, when the algorithms were tested on games of size 
2000 x 2000. Again, there is a clear split in the data between the “easy” games 
from GAMUT and the more challenging game classes. Note that, with only a 
handful of exceptions, the approximation algorithms were easily able to deal 
with games of this size. Only TS001 was observed to time out, and it timed out 
on 17% of the instances that it was tested on. This indicates that approximation 
algorithms can indeed be applied to games that cannot be tackled with exact 
methods. We summarize the performance of the algorithms as follows. 



A: % completed within 15mins B: e-Nash C: e-WSNE 
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for Blotto-4, 1999 for SGC, 200 rows for Tournament.) We display the average running time (in seconds) and average e-NE 
(e-WSNE for KS) across instances which did not time out (of the approximation algorithms only TS001 ever timed out - 17% 
of the time). 




DMP. This algorithm runs quickly and usually gives a poor approximation. In 
terms of quality, it is clearly outmatched by all of the other algorithms. 

BBM1 and BBM2. These algorithms were typically in the middle in terms of 
both approximation quality and running time. Only a handful of Blotto instances 
triggered the extra steps in BBM2, so these two algorithm are mostly identical. 
Pure. Whenever the game has a pure NE, this algorithm performs well, because 
it terminates once a pure NE has been found. Otherwise, it is among the slowest 
of the algorithms, because it is never faster than n 3 , where n is the number 
of strategies. The quality of approximation results confirm that our new games 
succeed in avoiding pure strategy profiles that are close to being Nash equilibria. 
TS2 and TS001. The TS algorithm was the clear winner in terms of quality 
of approximation. The results show that the choice of 6 can have a significant 
effect on the algorithm’s characteristics. TS2 often terminates in a reasonable 
running time when compared to BBM, and it usually beats BBM significantly 
on quality of approximation. However, TS001 always beats TS2 in quality of 
approximation, and always provides the best approximations among all of the 
algorithms that we studied. This accuracy comes at the cost of speed, as there 
are many games upon which TS001 is slower than TS2. 


Table 1 shows the results for the WSNE algorithms on the same set of games. 
Due to its 0(n 4 ) running time, we found that that KS+ timed out on all in¬ 
stances, so results for this algorithm are omitted. Recall that KS uses a prepro¬ 
cessing step to search for pure WSNE, and then solves an LP. We found that the 
preprocessing almost always provides the better approximation, but the search 
over pure strategies is the dominant component of KS’s running time. So, there 
is a significant cost for targeting e-WSNE over e-Nash equilibria, since Pure is 
the slowest algorithm for e-Nash equilibria, and KS is never faster than Pure. 


CovariantGame-09 
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Fig. 1: Runtime vs Quality of approximation for Covariant Game-9 and Ranking 
for instances of size 2000 x 2000. 




Finally, we comment on the trade off between running time and quality of 
approximation. In Figure 1 we plot these two metrics against each other for 
CovariantGame-9 and Ranking, which provide a fairly representative sample of 
the results that we observed across the dataset. The points towards the lower 
left of the diagrams are those that minimize the running time for a given ap¬ 
proximation guarantee. In order of accuracy, we typically see points from DMP, 
BBM1, TS2, and TS001 along this frontier. 

The TS algorithm. Since our results indicate that the TS algorithm gives the 
best approximations, it is worth spending more time analysing this algorithm. 
Both the quality of approximation and the running time of the algorithm are 
affected by the choice of <5. We now give more detailed results on how this 
parameter affects the of the algorithm. To test the dependence on 6 , we ran the 
TS algorithm on one hundred 400 x 400 instances of RandomGame for various 
values of S in the range (0, 0.14]. The results of these experiments are displayed 
in Figure 2. The left side of the figure shows the results for a single game, while 
the right side of the figure shows the average results over all instances. 

The first two rows show the runtime and quality of approximation, respec¬ 
tively. It can be seen that the algorithm does not scale smoothly with respect 
to S, and instead there are discontinuities in both running time and quality of 
approximation. The explanation for these discontinuities can be found in the 
third and fourth rows, which show the number of rows and the size of the LP 
that is solved in each iteration, respectively. The third row shows that, as we 
would expect, the number of iterations increases as S decreases. However, the 
data in the fourth row shows that the story is more complicated. The size of the 
LP that is formulated in each iteration increases as 6 increases. Thus, although 
the number of iterations falls, the time per iteration gets larger. 

4 Worst case examples 

The best theoretical upper bound on the performance of the TS algorithm is that 
it produces a 0.3393-Nash equilibrium, but previous work has not been able to 
show a matching lower bound. As we have seen, the algorithm usually finds a 
very good approximation in practice. The Blotto games were the only class that 
challenged the algorithm, and even then the approximations were usually good. 
Figure 3 shows box and whisker plots for the quality of approximation of TS001 
on the Blotto games that we considered. Almost all points were close to 0.01, and 
only a handful of instances were larger. The worst approximation that we found 
was a 0.14-Nash equilibrium, which is still far from the worst-case guarantee. 

To test the limits of the TS algorithm, we used a genetic algorithm to try 
and find worst-case examples. More precisely, we used a genome that encoded a 
5x5 game, and a fitness function that measured the quality of approximation 
found by TS001 on the game. The result of this was a 5 x 5 game 2 for which 
TS001 produces a 0.3385-Nash equilibrium. Note that this essentially matches 
the theoretical upper bound. 

Although this example provides a good lower bound against TS001, we found 
that BBM2 produced a 0.024-NE when applied to this instance. For this reason, 

All games found are given in the full version: http://arxiv.org/abs/1502.04980 
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Fig. 2: TS performance plots of runtime (row 1), quality of approximation (row 
2), number of iterations (row 3,) and average LP-Size (row 4) against <5. Left dia¬ 
grams shows results for a single 400 x 400 instance of RandomGame, while right 
diagrams show averages over one hundred 400 x 400 instances of RandomGame. 

we decided to test the limits of our entire portfolio of algorithms. We used the 
same genetic algorithm, but this time the fitness function was the best of the 
approximations found by TS001, Pure, and BBM2. We produced a 5 x 5 game 
for which all of the above algorithms produced at best a 0.3189-Nash equilibrium 
(TS produced a 0.3189-NE, Pure produced a 0.324-NE, and BBM2 produced a 
0.321-NE.) 

Both of the games mentioned so far contain dominated strategies. While 
this does not invalidate the lower bounds, it is obviously undesirable. For this 
reason, we reran the experiments with a fitness function that penalizes dominated 
strategies. For TS, we found a game with no dominated strategies for which the 
algorithm produces a 0.3254-NE, and for the portfolio we found a game with no 
dominated strategies for which the portfolio finds a 0.3253-NE. 

Finally, we applied the genetic algorithm to try and find a worst case example 
for KS+, but we were unsuccessful. We were able to produce a 5 x 5 game for 
which KS+ finds a 0.4799-WSNE, but this is not particularly useful, as some of 
the techniques used in this algorithm cannot possibly find an e-WSNE where e 
is better 0.5. 


5 Conclusion 

In this paper, we have conducted experiments to test the applicability of ap¬ 
proximation methods in practice. We found that the existing library of games 
provided by GAMUT is biased towards games that always have pure Nash equi¬ 
libria, so we introduced several new classes of games where this is not the case. 
Having done so, we are able to give conclusive answers to the questions that we 
posed in the introduction. Firstly, we have seen that the exact algorithms LH 
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Fig. 3: Box and whisker plots for quality of e-NE found by TS001 on Blotto 
games. 

and SE are quite limited in their ability to solve large games, particularly on the 
games that we have introduced. Secondly, in contrast to this, we have seen that 
approximation methods can tackle much larger instances, and that they provide 
approximate equilibria that are good enough to be practically useful. Finally, we 
have seen that DMP, BBM, and the variations of TS, can all be useful depend¬ 
ing on the quality of approximation that is required, which shows that there is 
a trade-off between running time and quality of approximation. In addition to 
this, we have also applied genetic algorithms to find a new worst case-example 
for TS which essentially match the theoretical upper bound. 

This work has highlighted the need for a comprehensive library of games upon 
which game theoretic algorithms can be tested. In addition to the games that we 
have introduced, there are also a number of other areas that could be represented 
here. For example, there are many auction problems from which games could be 
derived. Our study has focussed on algorithms with provable guarantees on the 
quality of approximate equilibria found in polynomial time. One direction for 
further study would to be to consider exact algorithms as heuristics for finding 
approximate equilibria. For example, one could randomly sample supports and 
find the best approximate equilibria on these supports using linear programming, 
or one could run an algorithm like LH for a fixed time or number of steps and 
check how good the strategy profiles it traces are as approximate equilibria. It 
would be interesting to see the extent that genetic algorithms can be applied 
to this. For example, what happens when we try to make LH bad, for quality 
of approximation or for running time? For approximation results, we would be 




interested in the approximation that is found after the algorithm has taken a 
fixed number of steps, like linear, quadratic, or some other polynomial. 


References 

1. Y. Anbalagan, S. Norm, R. Savani, and A. Vetta. Polylogarithmic supports are 
required for approximate well-supported nash equilibria below 2/3. In WINE, 
pages 15-23, 2013. 

2. H. Bosse, J. Byrka, and E. Markakis. New algorithms for approximate Nash equi¬ 
libria in bimatrix games. Theoretical Computer Science, 411(1):164-173, 2010. 

3. X. Chen, X. Deng, and S.-H. Teng. Settling the complexity of computing two-player 
Nash equilibria. Journal of the ACM, 56(3):14:1-14:57, 2009. 

4. B. Codenotti, S. D. Rossi, and M. Pagan. An experimental analysis of Lemke- 
Howson algorithm. CoRR, abs/0811.3247, 2008. 

5. C. Daskalakis, P. W. Goldberg, and C. H. Papadimitriou. The complexity of 
computing a Nash equilibrium. SIAM Journal on Computing, 39(l):195-259, 2009. 

6. C. Daskalakis, A. Mehta, and C. H. Papadimitriou. A note on approximate Nash 
equilibria. Theoretical Computer Science, 410(17): 1581 1588, 2009. 

7. J. Fearnley, P. W. Goldberg, R. Savani, and T. B. Sprensen. Approximate well- 
supported nash equilibria below two-thirds. CoRR, abs/1204.0707, 2012. 

8. N. Gatti, G. Patrini, M. Rocco, and T. Sandholm. Combining local search tech¬ 
niques and path following for bimatrix games. CoRR, abs/1210.4858, 2012. 

9. L. A. Goldberg, P. W. Goldberg, P. Krysta, and C. Ventre. Ranking games that 
have competitiveness-based strategies. Theor. Comput. Sci., 476:24-37, 2013. 

10. P. W. Goldberg, C. H. Papadimitriou, and R. Savani. The complexity of the 
homotopy method, equilibrium selection, and lemke-howson solutions. ACM Trans. 
Economics and Comput., 1(2):9, 2013. 

11. S. C. Kontogiannis and P. G. Spirakis. Well supported approximate equilibria in 
bimatrix games. Algorithmica, 57(4):653-667, 2010. 

12. S. C. Kontogiannis and P. G. Spirakis. Approximability of symmetric bimatrix 
games and related experiments. In SEA, pages 1-20, 2011. 

13. C. E. Lemke and J. Howson, J. T. Equilibrium points of bimatrix games. Journal 
of the Society for Industrial and Applied Mathematics, 12(2):pp. 413-423, 1964. 

14. E. Nudelman, J. Wortman, Y. Shoham, and K. Leyton-Brown. Run the gamut: 
A comprehensive approach to evaluating game-theoretic algorithms. In AAMAS, 
pages 880-887, 2004. 

15. R. Porter, E. Nudelman, and Y. Shoham. Simple search methods for finding a 
nash equilibrium. Games and Economic Behavior, pages 642-662, 2008. 

16. T. Sandholm, A. Gilpin, and V. Conitzer. Mixed-integer programming methods 
for finding nash equilibria. In AAAI, pages 495-501, 2005. 

17. R. Savani and B. von Stengel. Hard-to-solve bimatrix games. Econometrica, 
74(2):pp. 397-429, 2006. 

18. R. Savani and B. von Stengel. Unit vector games. CoRR, abs/1501.02243, 2015. 

19. H. Tsaknakis and P. G. Spirakis. An optimization approach for approximate Nash 
equilibria. Internet Mathematics, 5(4):365-382, 2008. 

20. H. Tsaknakis, P. G. Spirakis, and D. Kanoulas. Performance evaluation of a descent 
algorithm for bi-matrix games. In WINE, pages 222-230, 2008. 



Quality of Approximation 


A Appendix 


A.l Performance of TS 



Types of Blotto Games 


Fig. 4: Box and whisker plots for the quality of e-NE found by TS001. 



A.2 Close to worst case examples for TS 


A = 


‘56887 96209 18255 39791 34582" 
85196 65524 99782 97070 96765 
72792 22633 59840 99422 20117 
32963 18315 72881 85009 88426 
.26072 57677 19770 67548 43422. 


A = 


‘60599 68002 94170 91622 92353’ 
25814 90811 37672 74870 45424 
45917 33775 23776 45343 71345 
25339 61618 37098 24770 24292 
.39656 24016 63463 63396 75634. 


B = 


‘97209 43186 16342 16048 39059' 
60251 99413 3320 4596 28038 
39941 78448 52995 10725 76914 
91891 83644 53468 15871 16222 
.83130 99012 95399 95610 44184. 


B 


‘59772 98580 12901 34113 34916' 
98451 59776 20776 13893 27315 
17890 81316 36363 20947 36334 
53317 45329 14652 42939 17098 
.16562 67590 62024 16783 13882. 


(a) A bimatrix game for which TS001 finds (b) A bimatrix game for which TS001 finds 
a 0.3385-NE. a 0.3189-NE, Pure finds a 0.324-NE, and 

BBM2 finds a 0.321-NE. 


A = 


‘98433 55567 47993 40394 29895’ 
64392 81847 55756 40084 53038 
83294 65160 57977 29881 85410 
96761 64995 94650 66773 98497 
.49601 37356 53139 90306 40336. 


A = 


‘98433 55567 47993 40394 29895’ 
64392 81847 55756 40084 53038 
83294 65160 57977 29881 85445 
96761 64995 94650 66773 98497 
.49601 37356 53139 90306 40336. 


B = 


‘23881 47704 29961 29770 30574’ 
24025 29210 59730 62710 45525 
37004 26562 87859 87746 49475 
24815 25213 23425 94911 22725 
.89638 29302 95049 71163 41298. 


B 


‘23881 47704 29961 29770 30574’ 
24025 29210 59730 62710 45525 
37004 26562 87859 87746 49475 
24815 25213 23425 94911 22725 
.89638 29302 95049 71163 41298. 


(c) A bimatrix game without dominated (d) A bimatrix game without dominated 
strategies which TS001 finds 0.3254-NE. strategies which TS001 finds a 0.325-NE, 

Pure finds a 0.329-NE and BBM2 finds a 
0.374-NE. 



